Improving Chernoff criterion for classification by using the filled function

Linear discriminant analysis is a well-known matrix-based dimensionality reduction method. It is a supervised feature extraction method used in two-class classification problems. However, it is incapable of dealing with data in which classes have unequal covariance matrices. Taking this issue, the Chernoff distance is an appropriate criterion to measure distances between distributions. In the proposed method, for data classification, LDA is used to extract most discriminative features but instead of its Fisher criterion, the Chernoff distance is employed to preserve the discriminatory information for the several classes with heteroscedastic data. However, the Chernoff distance cannot handle the situations where the component means of distributions are close and leads to the component distribution overlap and underperforming classification. To overcome this issue, the proposed method designs an instance selection method that provides the appropriate covariance matrices. Aiming to improve LDA-based feature selection, the proposed method includes two phases: (1) it removes non-border instances and keeps border ones by introducing a maximum margin sampling method. The basic idea of this phase is based on keeping the hyperplane that separates a two-class data and provides large margin separation. In this way, the most representative instances are selected. (2) It extracts features on selected instances by the proposed extension of LDA which generates a desirable scatter matrix to increase the efficiency of LDA. In the proposed method, the instance selection process is considered a constrained binary optimization problem with two contradicting objects, and the problem solutions are obtained by using a heuristic method named filled function. This optimization method does not easily get stuck in local minima; meanwhile, it is not affected by improper initial points. The performance of the proposed method on data collected from the UCI database is evaluated by 10-fold validation. The results of experiments are compared to several competing methods, which show the superiority of the proposed method in terms of classification accuracy percentage and computational time.